Digital character interacting with customer in physical realm

ABSTRACT

Systems and methods for controlling performance of a digital character depicted at a display device are disclosed. According to at least one embodiment, a method for controlling performance of a digital character depicted at a display device includes: determining a presence of a person located in a physical environment; and in response to determining the presence of the person, facilitating control of the performance of the digital character depicted at the display device by a human operator, by an artificial intelligence (AI) game-engine, or by a combination thereof.

CROSS-REFERENCE TO RELATED APPLICATION(S)

Pursuant to 35 U.S.C. § 119(e), this application claims the benefit ofU.S. Provisional Patent Application No. 62/860,188, filed Jun. 11, 2019,the contents of which are hereby incorporated by reference herein in itsentirety.

BACKGROUND

Amusement parks may include various features to provide a uniqueexperience to guests. For example, an amusement park may have variousrides and shows that entertain guests. Furthermore, the amusement parkmay have show effects and props that may create a desirable environmentor atmosphere for guests. Such features may include entertainmentfigures (e.g., animated characters, animated figures) that may interactwith guests. For instance, the entertainment figures may speak, wave,walk, or perform any other suitable actions.

SUMMARY

With respect to various embodiments disclosed herein, techniques forcontrolling performance of a digital character depicted in digitalanimation are presented. According to various embodiments, theperformance is controlled to facilitate interaction between the digitalcharacter and a live customer.

According to at least one embodiment, a method for controllingperformance of a digital character depicted at a display device isdisclosed. The method includes: determining a presence of a personlocated in a physical environment; and in response to determining thepresence of the person, facilitating control of the performance of thedigital character depicted at the display device by a human operator, byan artificial intelligence (AI) game-engine, or by a combinationthereof.

According to at least one embodiment, an apparatus for controllingperformance of a digital character depicted at a display deviceincludes: a network communication unit configured to transmit andreceive data; and one or more controllers. The one or more controllersare configured to: determine a presence of a person located in aphysical environment; and in response to determine the presence of theperson, facilitate control of the performance of the digital characterdepicted at the display device by a human operator, by an AIgame-engine, or by a combination thereof.

According to at least one embodiment, a machine-readable non-transitorymedium has stored thereon machine-executable instructions forcontrolling performance of a digital character depicted at a displaydevice. The instructions include: determining a presence of a personlocated in a physical environment; and in response to determining thepresence of the person, facilitating control of the performance of thedigital character depicted at the display device by a human operator, byan AI game-engine, or by a combination thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects and features of the present disclosure willbecome more apparent upon consideration of the following description ofembodiments, taken in conjunction with the accompanying drawing figures.

FIG. 1 illustrates a system for controlling a digital character tointeract with a customer according to at least one embodiment.

FIGS. 2A and 2B illustrate examples of live-motion capture forcontrolling the digital character.

FIG. 3 illustrates an example of a rigid body according to at least oneembodiment.

FIG. 4 illustrates a 3D view of a capture area.

FIG. 5 illustrates an example delivery of a physical item for retrievalby the customer.

FIG. 6 illustrates a flowchart of a method of controlling performance ofa digital character depicted at a display device according to at leastone embodiment.

FIG. 7 is an illustration of a computing environment according to atleast one embodiment.

FIG. 8 is a block diagram of a device according to at least oneembodiment.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawing figures which form a part hereof, and which show byway of illustration specific embodiments of the present invention. It isto be understood by those of ordinary skill in this technological fieldthat other embodiments may be utilized, and that structural, as well asprocedural, changes may be made without departing from the scope of thepresent invention. Wherever possible, the same reference numbers will beused throughout the drawings to refer to the same or similar parts.

Embodiments disclosed herein are directed to a system for controlling adigital character to interact with a customer. For example, theinteraction may include a delivery of a physical product for retrievalby a customer. Example embodiments include a system that visuallyresembles a food truck and that outputs food for a customer, a systemthat vends movie tickets to a customer, and a system that facilitatesperformance of a rope tug-of-war between a digital character and acustomer.

An example embodiment will be described with reference to FIG. 1.

FIG. 1 illustrates a system 102 for controlling a digital character tointeract with a customer according to at least one embodiment. Thesystem 102 is located (at least in part) in an environment 100. Alsopresent in the environment 100 are one or more customers 110. The system102 includes a display device 104, and an output channel 108. Thedisplay device 104 is positioned to be in clear view of the customers110. As will be explained in more detail later, one or more items may bedelivered via the output channel 108 for retrieval by a customer 110.

Further, the system 102 includes at least one video capture device(e.g., a camera) and at least one audio capture device (e.g., amicrophone). The video and audio capture devices may be positioned, tocapture actions (or inactions) that occur in the environment 100—inparticular, actions (or inactions) made by the customers 110. Examplesof such actions include movements or motions made by a customer 110leading toward or away from the system 102 (e.g., toward or away fromthe display 104), motions or gestures made by the customer 110, andfacial expressions and reactions made by the customer 110. As willdescribed in further detail below, the capturing facilitates detectionof a presence of the customer 110 in the environment 100, and/orinteraction with the customer 110 (e.g., by one or more digitalcharacters depicted at the display device 104).

As noted earlier, the display device 104 of the system 102 is positionedto be in clear view of the customers 110. The display device 104displays video (e.g., digital video). As will be described in moredetail below, the video that is displayed may include interactiveelements and/or features.

For purposes of description, events that are depicted as occurring inthe video displayed at the display device 104 will be referred to asevents occurring “in the digital realm.” Separately, other events (e.g.,events occurring in the environment 100) will be referred to as eventsoccurring “in the physical realm.” From the perspective of the customers110, the display device 104 may be perceived as a “window” into aphysical space defined by the system 102, where events occurring in thedigital realm are perceived as occurring in a physical space defined byone or more elements of the system 102.

With continued reference to FIG. 1, a digital character 106 is depictedin the video that is displayed at the display device 104. By way ofexample, the digital character 106 may be authored via live-motioncapture of a human operator.

For example, with reference to FIG. 2A, the digital character 106 isauthored via live-motion capture of a human operator 202 located in anenvironment 200 a. The environment 200 a may be located in a vicinity ofthe environment 100, or at a location remote from the environment 100.If the environment 200 a is located near the environment 100, then theenvironment 200 a may be configured so that the human operator is hiddenfrom view of the customers 110 present in the environment 100.

In the environment 200 a, a display device 204 is provided. The displaydevice 204 outputs video and audio captured by the video and audiocapture devices that were described earlier with reference to the system102. Accordingly, the human operator 202 may be informed, in real-time,of actions (or inactions) that occur in the environment 100. In thismanner, the human operator 202 may effectively interact with thecustomers 110 present in the physical realm, via the digital character106 depicted in the digital realm.

For example, the human operator 202 may ask the customers 110: “Doesanyone want cookies?” Accordingly, through live-motion capture, thedigital character 106 is depicted in the digital realm as asking, “Doesanyone want cookies?” Customers 110 watching the digital video displayedat the display device 104 perceive such actions occurring in the digitalrealm.

The customers 110 may respond accordingly. For example, one customer 110may raise his/her hands. As another example, another customer 110 mayexclaim: “Yes!”

By monitoring the display device 204, the human operator 202 is able toobserve such actions made by the customers. For example, by continuingto monitor the camera footage, the human operator 202 can (1) change orshift his/her gaze such that the digital character 106 looks directlyinto the eyes of a customer 110 (e.g., if the customer 110 remainsrelatively still), and (2) reach towards the customer 110 (e.g., if thecustomer 110 reaches towards or points to a particular portion of thedisplay device 104). In a similar manner, other interactions between thedigital character 106 and the customer 110 may include a shaking ofhands with the customer 110, a hugging of the customer 110, beingtangibly affected by a push from the customer, etc.

In at least one embodiment, the bridging of the digital realm and thephysical realm may be achieved by configuring the digital character toextend out a limb for shaking a hand of the customer 110, or for givingan item to the customer 110. By way of example, this may be achieved bycontrolling a physically operated puppet limb, operated in the physicalrealm by a human puppeteer who is hidden from view of the customers 110.The physically operated puppet limb may be controlled to reach outtowards the customers 110 and directly interact with the customer 110 inthe physical realm. In this situation, the physically operated puppetlimb may be an animatronic arm and hand that moves realistically basedon real-time control by the puppeteer. According to another embodiment,a digitally operated puppet limb may be controlled to reach out anddirectly interact with the customer 110. For example, the digitallyoperated puppet limb may be controlled via digital information from aperformer (e.g., human operator 202) who is authoring the characterexperience. In this situation, the performer may control the limb inreal-time to extend or retract, by manually operating an onscreeninterface, a keyboard trigger, a button or a joystick input. Real-timedata from the performer would operate the mechanics of the hand andcause the arm to move around in a controlled, robotically legal manner.According to another embodiment involving a digitally operated puppetlimb, the performer may instigate the handshake simply by reachinghis/her arm out to a known area of a capture volume. The extending andretraction of the puppet limb and other movements thereof would besolved by the performer, with the results also retargeted to theanimatronic puppet limb in a controlled, robotically legal way.

In response to seeing and/or hearing reactions made by the customers110, the human operator 202 may move within the environment 200 a, as ifhe were retrieving a box of cookies for the customers 110. Then, withreference to FIG. 2B, the human operator 202 may move within theenvironment 200 b as if he were delivering the retrieved box to one ormore waiting customers 110.

The movements of the human operator 202 are replicated in the digitalrealm by the digital character 106. For example, in the digital realm,the digital character 106 retrieves a box of cookies and then moves in adirection leading towards the output channel 108.

In the foregoing, the movements of the human operation 202 may bemonitored using a motion capture system. Motion capture systemsaccording to one or more embodiments will now be described in moredetail.

Optical systems (or optical motion capture systems) utilize datacaptured from image sensors to triangulate the 3D position of a subjectbetween two or more cameras calibrated to provide overlappingprojections. The data may be captured using markers attached to an actor(e.g., human operator 202). Tracking a larger number of performers orexpanding a capture area may be achieved by increasing a number ofcameras.

With respect to optical systems, occlusions occur when a model markerpoint is not present in a cloud of points, for example, when the numberof observed points is lower than the number of model marker points. Suchevents may occur due to any of variety of reasons, such as theinterposition of one or more opaque objects between the marker and thecameras or when the marker goes out of the camera field of view (FOV).

According to at least one embodiment, a drift corrections systemincludes (e.g., in a hybridized manner) one or more aspects of anoptical motion capture system, together with one or more aspects of aninertial motion capture system. Such a system may allow forocclusion-free motion capture that is correct positionally in a capturearea (such as environment 200 a) and that would not drift over time.

Aspects of the inertial motion capture system include using an inertialbody capture suit as known in the art. The inertial body capture suitmay be of a type that is for facilitating live-motion capture of thehuman operator 202, as described earlier with reference to FIG. 2A. Forexample, such an inertial body capture suit measures the joint angles ofthe human operator 202 and estimates his/her position from an initialstarting point, for example, by counting footsteps. In this regard,errors may accumulate over time, because the inertial motion capturesystem produces approximate results based on magnetometers that areincluded in the suit. For example, the inertial motion capture systemmay approximate a location of true north, but errors may result due toeffects of natural magnetic interference on the accuracy of themagnetometers.

To help correct for phenomena such as occlusions and artifacts such asthose caused by magnetic interference, a system may use aspects of bothan inertial motion capture system and an optical system. According to atleast one embodiment, the aspects of the optical system would be simplerthan those of an optical system that might typically be used for fullbody motion capture. For example, the optical system may be used tofocus on the waist of the actor. According to at least one embodiment,an optical system includes using a rigid apparatus (or rigid body) thatis wearable by the human operator 202. According to at least oneembodiment, data from an inertial motion capture system and data from anoptical system are combined (or fused together) and used to drive adigital character (e.g., digital character 106).

By way of example, data regarding rotations and height off the ground(see, e.g., Y-axis of FIG. 2A) may be received from an inertial bodycapture suit of an inertial motion capture system, and data regarding ahorizontal plane (see, e.g., a plane defined by X- and Z-axes of FIG.2A) may be received from an optical system.

As described earlier, an optical motion capture system may include arigid body. The rigid body may be stably attachable to the humanoperator 202 being drift corrected. According to at least oneembodiment, the rigid body includes markers placed thereon. For example,the markers may be retro-reflective markers. Such markers may be placedon the rigid body in a unique pattern such that the rigid body wouldhave a unique identity (e.g., configuration or arrangement of markers)that is recognizable or identifiable by the optical motion capturesystem.

FIG. 3 illustrates an example of a rigid body 502 according to at leastone embodiment. As illustrated, the rigid body 502 may take the form ofa belt. The belt is configured to be worn around the waist of a person(e.g., human operator 202). The rigid body 502 is relatively small inbulk, such that, while the wearer of the rigid body moves, the rigidbody moves with the hips of the wearer without moving independently withrespect to the wearer. The rigid body 502 may be made of a fairly rigidmaterial, and may have a uniform width (e.g., 1 or 2 inches). Markers506 may be positioned along the circumference of the rigid body 502. Asdescribed earlier, the markers 506 may be positioned on the rigid bodyin a unique pattern. For example, in the example illustrated in FIG. 3,a total of twenty-five markers 506 are placed along the circumference ofthe rigid body 502 in a unique zigzag pattern.

Alternatively, instead of positioning markers 506 directly on a belt,the markers 506 may be positioned (indirectly) on a belt via one or morerigid bodies that are positioned (directly) on the belt. In thisalternative example, markers 506 may be placed directly on the rigidbodies, which are, in turn, placed on the belt. For example, fivemarkers 506 may be placed on each of a total of five rigid bodies, whichare, in turn, placed on the belt. The placement of the rigid bodies onthe belt (as well as the placement of the markers on the rigid bodies)may be in a unique pattern.

It is understood that as few as one optical marker may be sufficient forpurposes of achieving an acceptable level of drift correction. However,according to at least one embodiment, two or more optical markers areutilized to achieve a higher level of robustness with respect toocclusion.

With respect to the optical system, various embodiments will now bedescribed with reference to an “extrinsic” system. According to suchembodiments, an array of two or more cameras is installed around theperiphery of a capture area (e.g., a room). The cameras capture views ofthe capture area as well as of one or more markers (e.g., markers 506 onthe rigid body 502) as the rigid body (as worn by a human operator)moves around the capture area. The static cameras observe the movingmarkers and, accordingly, provide data (e.g., accurate rotations andtranslations) regarding the moving markers, solved to a rigid object(e.g., a computer-generated bone) in computer-animation software.

FIG. 4 illustrates a 3D view of a capture area (e.g., room) 400. In atleast one embodiment, one or more cameras are installed on each towerthat is placed in the capture area 400. For example, as illustrated inFIG. 4, four towers 402 are placed in the capture area 400, one tower402 being located at each corner. For purposes of illustration, eachtower 402 may be an eight-foot-tall box truss including a two-foot-wide,one-inch-thick aluminum plate. Each of the towers 402 may have threecameras 406 installed thereon. Within one tower 402, the cameras 406 maybe spaced apart from each other—e.g., by approximately two feet betweenone another along the height of the tower.

The cameras 406 may be on a small aluminum plate that is inserted intothe box truss. A geared three-way positioning head may also be on theplate. The cameras 406 may be attached to the positioning head, and thehead may be geared to be set at exact rotations and to be locked intoplace once aiming of the head is completed. As noted earlier, the towers402 are positioned at the corners of the capture area 400.

Each tower 402 generates its own data. During calibration of the cameras406, a static object (e.g., rigid body 502) may be placed at a middle ofthe capture area 400. The cameras 406 may be calibrated concurrently byconfiguring the optical system to triangulate off of the static object.Accordingly, a position of the static object within the room may bedetermined. As noted earlier, data regarding a horizontal plane (see,e.g., a plane defined by X- and Z-axes of FIG. 2A) may be received fromthe optical system. Such data may be combined (or fused) with dataregarding rotations and height off the ground (see, e.g., Y-axis of FIG.2A), which is received from an inertial body capture suit of an inertialmotion capture system. A combination of the data may be used to generateanimation of a digital character. For example, rotational data receivedfrom the inertial system may be used to animate a skeleton of thedigital character, and data received from the optical system may be usedto correct a position of the hips of the digital character.

With respect to the optical system, various embodiments have beendescribed with reference to an “extrinsic” system. According to at leastanother embodiment, an “intrinsic” system is utilized. In such a system,a camera is worn by a human operator to monitor markers located in thecapture area (e.g., over the ceiling, the floor and/or walls of theroom) according to a unique pattern. When the human operator moves aboutthe room, the camera moves together with the human operator and monitorsthe static markers such that an accurate estimate of its own rotationand position in the room, relative to the fixed markers on the walls,floor or ceiling of the room, may be produced.

As described earlier, markers placed on a rigid body and/or a belt maybe retro-reflective markers. According to at least one other embodiment,the markers may be light emitting devices (LEDs) that emit light in awavelength detectable by the optical motion capture system.

As described earlier, a drift corrections system is created by combining(or hybridizing) aspects of an optical motion capture system, withaspects of an inertial motion capture system. According to at least oneother embodiment, a drift corrections system is created by combining (orhybridizing) aspects of a magnetic motion capture system, with aspectsof an inertial motion capture system. Such embodiments may operate in amanner similar to the manner described earlier with respect toembodiments employing an optical motion capture system. For example, amagnetic “marker” may be placed on a human operator. At either end ofthe room, a magnetic detection system may be placed. The magneticdetection system may emit a magnetic field and detect a magneticinterference caused to the field by the magnetic marker. The detectedinterference may be used to determine the position of the magneticmarker.

In at least one embodiment, techniques employing Wi-Fi technologies maybe utilized. Such embodiments may operate in a manner similar to themanner described earlier with respect to embodiments employing amagnetic motion capture system. For example, a Wi-Fi antenna or antennasmay be placed around the room, and a Wi-Fi emitter may be placed on thehuman operator. As the human operator moves within the room, the Wi-Fiemitter may move closer to or farther from one antenna or another.Characteristics of the signal received at the antenna(s) may be used todetermine the position of the emitter within the room.

In at least one embodiment, techniques employing acoustic technologiesmay be utilized. Such embodiments may operate in a manner similar to themanner described earlier with respect to embodiments employing anoptical motion capture system. In this regard, one of at least twoconfigurations may be used. For example, in one configuration, amicrophone is placed on the human operator, and ultrasonic emitters areplaced around the room. As another example, in another configuration, anemitter is placed on the actor, and microphones are placed around theroom. In embodiments employing acoustic technologies, time of flight(ToF) may be used to determine the position of whichever object isplaced on the human operator (e.g., microphone or emitter). Much in theway echolocation works, the emitters make an ultrasonic ping sound andthe microphone (or microphone array) detect the ping and calculate thedistance traveled by the ping sound based on the delay (the time offlight) from emitter to microphone.

In at least one embodiment, one or more depth cameras may be utilized todetermine the position of the human operator in the room. For example, adepth camera may project an infrared (IR) grid and then detect how thehuman operator is distorting that grid to determine where the operatoris and movements made by the operator. As another example, an array ofIR LEDs may flash on and off in sequence to perform ToF calculationswhen light shines from the LEDs, bounces off the human operator and thenreturns to the depth camera, to make the depth calculations.

In at least one embodiment, markers need not be employed. For example,in a simpler system, one or more video cameras may be used to trackparticular features of the human operator or on the rigid body, todetermine the position of the human operator in the room.

With reference to an environment 300 of FIG. 5, in the physical realm, abox 302 containing cookies is delivered at the output channel 108. Forexample, the system 102 may include a dispenser that dispenses itemsupon being controlled (e.g., by the human operator 202).

As described earlier with reference to FIG. 1, the system 102 includesat least one video capture device (e.g., a camera) and at least oneaudio capture device (e.g., a microphone). It is understood that thecamera and microphone and camera can exist in the virtual world and beexperienced by a customer in virtual reality. For example, a customermay be controlling a digital avatar existing in the digital realmthrough an input device, such as a joystick, a mouse, or a virtualreality (VR) or motion capture input. Interactions occur within thedigital realm, except for a vending/delivery of an item (e.g., box 302),which still occurs in the physical realm. For example, the customerplays a game, which interacts with game characters corresponding to thecustomers, and not directly with the customers. In this situation, thecustomers enter and experience the digital realm as digital avatars, themovements of which are controlled by the customers, based on inputprovided in the physical realm. As such, an avatar can be controlled tomove in the digital realm in some manner (e.g., walk, jump, fly,teleport). When the presence of the digital avatar is detected in thedigital realm, interactions with the digital avatar are initiatedsimilar to a manner in which interactions with a customer are initiatedupon detection of a presence of the customer in the physical realm(e.g., environment 100). Once the customer has triggered a change tooccur in the physical realm (e.g., the delivery of an item), the changeis realized in the physical realm.

As also described earlier with reference to FIGS. 1, 2A, 2B, 3 and 4, adigital character (e.g., digital character 106) may be authored vialive-motion capture of a human operator (e.g., human operator 202).Live-motion capture driven operation may be applied to a particular part(or aspect) of the digital character (or an object such as box 302 inFIG. 5), or may drive the entire digital character or prop, whensuitable. According to one or more embodiments, the human operatordrives the performance of the digital character with real-time MusicalInstrument Digital Interface (MIDI) input and/or digital puppeteeringusing devices such as controllers, keyboards, joysticks, mice, footpedals, microphones, etc. In such embodiments, live-motion capture maynot be used exclusively. For example, motion capture may be used todrive the performance of the face of a digital character, and theremaining aspects of the performance are driven using triggered inputdevices. According to one or more other embodiments, the digitalcharacter 106 may be authored entirely via live-motion capture of ahuman operator.

According to one or more other embodiments, the performance anddecision-making for the digital character is authored entirely by anartificial intelligence (AI) character resident in a game engine.

According to one or more other embodiments, the performance anddecision-making for the digital character may be achieved using acombination of an AI game-engine driven character and human motioncapture. By way of example, during particular times(s)—for example,while the system is waiting to be engaged by a customer—the digitalcharacter may be at least partially driven by AI (e.g., by playing loopsof ambient motion or other character-based animation). At anothertime(s), a transition from being driven by AI to being driven by motioncapture may occur. For example—upon system detection of a presence of acustomer 110, the performance (e.g., animation control) of the digitalcharacter may be transitioned to the human operator, who can interactwith the customer in a more personable manner. Leading to such atransition, the system 102 may signal the human operator that he/she isto be given at least partial control over the performance of the digitalcharacter. For example, the system 102 may provide an audio and/orvisual indication(s) to inform the human operator that he/she is to begiven control over the vocal performance of the digital character, thefacial performance of the digital character, or the entire performanceof the digital character.

In a similar manner, a transition from being driven by motion capture tobeing driven by AI game-engine technology may occur. Decisions made bysuch AI technology may be based on analysis of data captured by thevideo capture device (e.g., camera) and/or the audio capture device(e.g., microphone) of the system 102. For example, analysis of datacaptured by the camera may be used to identify not only positions ofvarious customers 110, but also certain characteristics of particularcustomers, for example, a blue-colored article of clothing (e.g., shirt)worn by one customer 110, or the blond-colored hair of another customer110. As such, even when the performance of the digital character isbeing driven by AI game-engine technology (rather than by a humanoperator), the digital character may still, to some degree, interactwith the customers 110. For example, the eyes of the digital charactermay be driven to move such that it appears that a “gaze” of the digitalcharacter follows movements made by a customer 110. Also for example,the digital character may be driven to wave at a particular customer 110and/or say a somewhat customized or personalized greeting to thecustomer 110 wearing the blue article of clothing or the customer 110who has the blonde hair.

Similarly, analysis of data captured by the microphone may be used tointerface with the customers 110, at least to some degree. For example,when such analysis detects a sound as having a particular characteristic(e.g., a sound exceeding a certain threshold audio level), theperformance of the digital character may be selectively driven by aparticular clip of (prepared) animation. In such a clip, the eyes of thedigital character may move such that it appears that the gaze of thedigital character shifts up and outwards, as if in search of the sourceof the sound that was heard. As another example, waveform analysis maybe performed on speech audio that is captured by the microphone. Suchanalysis may be used in discerning the mood or emotional state of acustomer 110 that has spoken. In this situation, the mood in which thedigital character delivers an audio greeting to the customer 110 may beselected to match or mirror the mood that is discerned by the waveformanalysis. For example, if the mood of the customer 110 is discerned asbeing happy or cheerful, the digital character may be controlled toaddress the customer 110 in a happy or cheerful manner.

Accordingly, AI game-engine technology may be used to effectivelyexecute a certain branch in a decision tree (e.g., using the selectedclip of animation to drive the performance of the digital character) inresponse to a certain catalyst (e.g., detection of a sound having aparticular characteristic). This may improve the likelihood that thedigital character appears lifelike and capable of reacting to real-worldevents occurring in the physical realm (e.g., of the environment 100).This may be preferable to performance of a digital character in whichthe character merely repeats certain actions, while appearing to beoblivious to events occurring in the physical realm.

As such, the performance of the digital character need not be driven bya human operator at all times. For example, the performance of thedigital character may be driven by AI game-engine technology during some(or most) times, and by a human operator during selected times (e.g.,one or more key times). As another example, the performance of thedigital character may be driven by AI game-engine technology during mostof the day (e.g., 11 hours, in total, over a 12-hour day of operation),and control of the digital character may be assigned to a human operatorduring a selected period (e.g., a selected hour during such a day ofoperation). The period may be selected in order to provide morepersonalized interaction with the customers 110 during that period. Atthe end of such a period, the human operator may return control of theperformance of the digital character to the AI game-engine technology.The return may be chosen to occur at a time so as to provide a moreseamless transition, from the perspective of the customers 110. Thehuman operator may return control of the performance of the digitalcharacter at such a time by, for example, manually operating an onscreeninterface, a keyboard trigger, a button or a joystick input.

In at least one embodiment, performance of the digital character may, attimes, be driven concurrently by AI game-engine technology as well as bya human operator. For example, when the digital character is at leastpartially driven by AI in interacting with a customer 110, a humanoperator may wish to enhance the performance of the digital character.In this situation, the human operator may control the AI-drivenperformance in one or more ways. For example, if the human operatordiscerns the mood of the customer 110 as being not merely happy orcheerful, but, rather, particularly happy or cheerful, the humanoperator may control the digital character to address the customer 110in a similarly heightened manner. Such control may be executed by, forexample, manually operating an onscreen interface, a keyboard trigger, abutton or a joystick input. In this manner, the performance of thedigital character may (temporarily) be directed, at least in part, bythe human operator.

According to embodiments that will be described in more detail later,multiple digital characters may be depicted at the display device 104.Each of such characters may be controlled via a respective varyingcombination of AI game-engine driven character and human motion capture,as described earlier.

As described earlier with reference to various embodiments, actions (orinactions) that occur in the environment 100—in particular, actions (orinactions) made by the customers 110—are captured, e.g., by a camera anda microphone. Upon detection of a presence of a customer 110 in theenvironment 100 (e.g., one or more particular areas within theenvironment 100), interactions with the customer are initiated. It isunderstood that the detection may be performed, additionally and/oralternatively, using other devices. Such other devices include apressure pad, a depth camera, a sonic-range detection device, alight-beam device, and/or a heat- or sound-detection device.

The devices may detect not only the presence of the customer, but alsoother parameters such as the head height and the body position of thecustomer. For example, the customer may be detected using video cameras,depth sensors, or a light or sound beam that is broken, to determine thebody position and the height of the customer. Knowledge of suchinformation (e.g., in relation to the position of the display device104) may allow angular corrections to be made to the head or eye anglesof the digital character 106, so that the eye line of the digitalcharacter aligns more closely with that of one or more of the customers.By detecting a head height of the customer more accurately, such devicescan effectively “true up” the eye line of the digital character.

As also described earlier with reference to various embodiments, theinteraction includes one or more interactions between a digitalcharacter and a customer with respect to both the digital realm and thephysical realm. For example, in the digital realm, the digital charactermay perform steps to prepare or procure an item (e.g., a box 302 ofcookies) for a customer. In the physical realm, the item is given (orsold) to the customer in real-time, such that the item is capable ofbeing retrieved or collected by the customer (e.g., at the outputchannel 108). According to one or more further embodiments, the item isprovided such that it can be retrieved by the customer at anotherlocation (e.g., somewhere outside of the environment 100). According toone or more further embodiments, the item is delivered to the customer(e.g., by physical mail) at a later date and/or time.

According to one or more other embodiments, the interactions between thedigital character and the customer include an interaction that causes(or brings about) a change in the physical realm of the environment 100.For example, the system 102 may include a pulley to facilitateperformance of a game of tug-of-war between the digital character andthe customer.

By way of example, a human operator 202 having control of theperformance of the digital character asks: “Does anyone want to play agame of tug-of-war with me?” Accordingly, in the digital realm, thedigital character 106 asks the customers 110: “Does anyone want to playa game of tug-of-war with me?” In the physical realm, the customers 110may respond accordingly. For example, one customer 110 may raise his/herhands. As another example, another customer 110 may exclaim: “Yes!”

In response to seeing and/or hearing reactions made by the customers110, the human operator 202 may move within the environment 200 a, as ifthe human operator 202 were retrieving a rope with which the game is tobe played. Then, the human operator 202 may move within the environment200 b as if the human operator 202 were moving the retrieved ropetowards one or more waiting customers 110.

The actions of the human operator 202 are replicated, in the digitalrealm, by the digital character 106. For example, in the digital realm,the digital character 106 retrieves a rope and moves in a directionleading towards the output channel 108.

In the physical realm, a free end of a rope is extended out through theoutput channel 108. The other end of the rope may be coupled to a pulleysystem that is hidden from view of the customers 110. After one of thecustomers takes hold of the rope, a game of tug-of-war may begin.

According to one or more further embodiments, examples of otherinteractions that cause a change in the physical realm of theenvironment 100 include: an object in the physical realm being caused tobe moved (e.g., knocked over) in response to an action (e.g., trippingor falling) by the digital character in the digital realm; a light inthe physical realm being controlled to be turned on/off in response toan action by the digital character in the digital realm; and some otheraspect in the physical realm to be changed in response to an action bythe digital character in the digital realm.

In a similar manner, actions by a customer in the physical realm thatare detected by the system 102 may result in a change in the digitalrealm. For example, facial responses or expressions by a customer, thebody posture of a customer, other movements made by a customer, or vocalreactions or sounds made by a customer may be detected and cause relatedevents to occur in the digital realm.

The interactions between the physical realm and the digital realm arenot limited to those involving customers and digital characters. Forexample, the presence of one or more specific objects in the physicalrealm that is detected may result in a change in the digital realm, andvice versa. For example, the change may involve corresponding or relatedobjects in the digital realm. By way of example, an object being movedby the customer in the physical realm, either intentionally orunintentionally, may be followed by a movement of a corresponding orrelated object by the digital character in the digital realm.

Causing a change in the physical realm of the environment 100 may beperformed to increase the likelihood that the customer will engage withone or more digital characters in the digital realm. If customerengagement is not detected and the performance of the digital characteris at least partially authored by AI, the system 102 may enter a loopstate where it continuously (or periodically) monitors for detectedengagement. Alternatively (or in addition), the system 102 may begin todisplay follow-up animations in order to motivate the customer to engagewith the digital character(s). Alternatively (or in addition), thesystem 102 may return to a waiting state in which a particular set ofanimations is displayed until the presence of a customer is detected.

As described earlier with reference to various embodiments, an objectsuch as a box of cookies may be delivered (e.g., via the output channel108) for retrieval by a customer. In this situation, the object may bedelivered relatively soon after the request by the customer (e.g., thecustomer says “Yes!”) is detected. In other situations, the delivery mayoccur after a certain delay. For example, a toy sword may be deliveredto the customer. In the physical realm, a pre-made toy sword may beretrieved for placement at a receiving end of the output channel.However, in the digital realm, a digital character may prepare the toysword (e.g., from raw materials) in a blacksmith shop. In such asituation, animation loops (or sequences) may be displayed at thedisplay device (e.g., display device 104), to show the preparation ofthe toy sword. The display of such loops may continue, until feedback isreceived in the physical realm indicating that delivery of the toy swordis to occur. Upon reception of such feedback, the display of the loopscomes to an end, and, in the digital realm, the preparation of the toysword is completed. Examples of such feedback will be described in moredetail later.

According to another example, the object may be an item that doesrequire some preparation time in the physical realm. For example, theobject may be a food item (e.g., a bowl of soup noodles) that isrequested (or ordered) by the customer.

Before the request is received, the presence of the customer may firstbe detected. As described earlier with reference to various embodiments,movements of one or more customers (e.g., in an environment 100) may bedetected. Upon the detection of one or more movements (e.g., one or moreparticular movements such as movements towards the display device 104),actions are performed to solicit engagement by the customer. Forexample, when the system 102 is designed to visually resemble a kitchenor the exterior of a food truck, a sequence (e.g., game sequence) isinitiated such that the display device 104 displays one or more digitalcharacters that are preparing food. For example, in the digital realm,the digital characters are chopping vegetables, cooking noodles, etc.

The depiction of one more items in the digital realm, such as a spoon orpot for cooking, can be controlled in a number of ways. For example, theitem may have a counterpart present in the physical realm, such as aspoon (made, e.g., out of plastic or foam) that is carried by an actor(e.g., human operator 202 of FIG. 2A) who can be tracked by motioncapture. As another example, the depiction of the item could becontrolled digitally, when the item exists within a game engine like anobject in a video game. In this situation, an actor may interact withthe item by making a grabbing gesture and putting his/her hand near theitem to alert the game engine that he/she wants to pick up the item. Theactor may also trigger the grabbing gesture via a controller or buttonpress. In another example, the actor could trigger the item to appearfrom thin air into his/her hand by pressing a button or by reaching intoa specific ‘zone.’

As described earlier with reference to FIG. 2B—in the digital realm, thedigital character 106 retrieves a box of cookies. The actor may achievethis retrieval of the box by reaching his hand up into the air so thatthe hand of the digital character 106 enters into an invisible cube inthe digital realm that exists out of frame (e.g., just out of frame)with respect to a display (e.g., display 104). Once the hand of thedigital character 106 has entered this zone, a box is placed in his handin the digital realm. When the actor lowers his arm in the physicalrealm, the box appears in the hand of the digital character 106 in thedigital realm, appearing as if the digital character 106 had reachedoffscreen and pulled a box from an unseen shelf.

According to another example, a combination of one or more items in thephysical realm along with one or more items in the digital realm isutilized. For example, such a combination may be utilized to achieve thechopping of a carrot in the digital realm by the digital character. Thecombination may involve a counterpart (e.g., a physical prop) in thephysical realm that is held by the actor and that corresponds to a knifein the digital realm, and a carrot existing only in the digital realm.Movement of the physical prop is tracked such that the knife appears inthe hand of the digital character and moves accordingly. As the cuttingedge of the knife moves near the carrot in the digital realm, the carrotis depicted as being sliced into pieces. As the side of the knife isbrought near the carrot pieces, the pieces may be depicted as beingmoved off the edge of a cutting board and into a pot. This movement ofthe carrot pieces may be depicted as also being caused directly by thehand of the digital character (for example, by the actor motioning so asto sweep the carrot pieces into the pot). However, in order to achieve ahigher degree of realism, it is understood that only the knife (and notthe hand of the digital character) is capable of causing the carrot tobe sliced into pieces.

Triggered motion(s) can also be achieved in a similar manner. Suchmotions include, for example, martial-arts movements that a typicalunskilled person cannot readily perform. These motions may be triggeredby a button press on a controller that blends from live motion into oneor more canned motions or may be triggered by the actor performing agesture (e.g., an alternative gesture) that, when recognized by thesystem, launches the triggered motion. For example, an actor could kickhis/her leg, and the system may recognize the kick as a trigger for aspecial Kung Fu jump-kick sequence. According to a further example, thetriggered motion does not completely drive the performance of thedigital character. For example, only one or more portions of the body ofthe digital character may be driven by the triggered motion. By way ofexample, while the triggered motion is performed by the torso, armsand/or legs of the digital character, the face of the digital charactermay still be controlled by the actor (e.g., via motion capture).

Returning to detection of a presence of a customer, a request from thedetected customer is received. Receiving the request may involve usingnatural language processing (to receive and process verbal requests).Alternatively (or in addition), receiving the request may involve usingan interface that is operable by the customer. For example, the customermay operate a button on a touchscreen, or fill out an instruction cardthat is inserted into and read by a machine.

After the customer requests a specific item, preparation of the itembegins in the physical realm. For example, a food preparer (e.g., acook) who is located in the vicinity of the environment 100 beginspreparing the item that was requested.

Concurrently, in the digital realm, actions depicting the preparation ofthe item occur. Such actions may be customized based on the customer'srequest. For example, based on the customer's request that the soupnoodles include extra carrots, a sequence that is executed may includedepiction of carrots flying across the kitchen of the digital realm.

The animation that is displayed at the display device 104 may becontinuously controlled while preparation of the requested item in thephysical realm is in progress. For example, during preparation of therequested item, feedback may be received from the food preparer, e.g.,indicating that more time is required before the preparation isfinished. Such feedback may cause additional loops of a particularsequence to be displayed. Here, it is understood that the animation thatis displayed at the display device 104 may be controlled concurrently byAI game-engine technology as well as by a human operator, similar to amanner in which the performance of a particular digital character may becontrolled, as described earlier with reference to various embodiments.

Additionally, the animation may continue to be controlled when therequested item is nearing or at completion. For example, feedback may bereceived from the food preparer, e.g., indicating that the item is readyfor output. The feedback may originate from a pressure sensor thatsenses that an object (e.g., the requested item) has been placed at adetermined location. Accordingly, the food preparer may place therequested item at or near the pressure sensor, to provide feedbackindicating that the item is ready for output. As another example, thesystem 102 may include an interface that is operable by the foodpreparer. Accordingly, the food preparer may operate a button on atouchscreen to signal that the item is ready for output.

In the physical realm, the item may be placed at the output channel 108for retrieval by the customer.

Based on features described, the timing of events occurring in thedigital realm (e.g., the preparation of the item by the digitalcharacter(s) as depicted the displayed animation) can be better alignedwith the timing of events occurring in the physical realm (e.g., thepreparation of the item by the food preparer). For example, the timingin both realms can be better aligned, such that the bowl of soup noodlesappears in the physical realm at the output channel 108, as itdisappears from view in the digital realm. According to one or moreparticular embodiments, the item presented at the output channel 108 isselected to match in visual appearance the item that is depicted in thedigital realm.

FIG. 6 illustrates a flowchart of a method 600 of controllingperformance of a digital character depicted at a display deviceaccording to at least one embodiment.

At block 602, according to a particular embodiment, the performance ofthe digital character may be controlled using AI game-engine technology.For example, with reference to FIG. 1, the performance of the digitalcharacter 106 is controlled using AI game-engine technology.

At block 604, a presence of a customer located in a physical environmentis detected. For example, with continued reference to FIG. 1, a presenceof a customer 110 located in the physical environment 100 is detected.

According to a further embodiment, the detection of the presence of thecustomer is performed based on data received from at least a camera or amicrophone located in the physical environment.

According to a further embodiment, detecting the presence of thecustomer may include autonomously determining a characteristic of anappearance of the customer or an emotional state of the customer. Forexample, data received from the camera may be used in determining that acustomer 110 is wearing a blue-colored article of clothing. As anotherexample, data received from a microphone may be used for detection andfor determining that the customer 110 is happy or sad.

At block 606, in response to detecting the presence of the customer,control of the performance (or at least an aspect thereof) of thedigital character by a human operator, an AI game-engine, or acombination thereof is facilitated. For example, with reference to FIG.2A, in response to detecting the presence of the customer 110, controlof the performance of the digital character 106 by a human operator 202is facilitated.

According to a further embodiment, facilitating control of theperformance of the digital character may include providing at least oneoption selectable by the human operator. The selectable option is forcontrolling the digital character to address the customer according tothe determined characteristic or the determined emotional state. Forexample, if the human operator 202 discerns the mood of the customer 110as being particularly happy or cheerful, the human operator may controlthe digital character 106 to address the customer 110 in such aparticularly happy or cheerful manner.

According to a further embodiment, facilitating control of theperformance of the digital character may include receiving motioncapture data corresponding to the human operator. By way of example,receiving the motion capture data may include receiving data from anoptical motion capture system (e.g., a system utilizing cameras 406 ofFIG. 4) and receiving data from an inertial motion capture system (e.g.,a system utilizing an inertial body capture suit worn by the humanoperator 202).

According to a further embodiment, the data received from the opticalmotion capture system may include data corresponding to one or moreoptical markers located at a waist region of the human operator. Forexample, the data received from the optical motion capture system mayinclude data corresponding to markers 506 that are located at a waistregion of the human operator 202.

In a further embodiment, the control of at least the portion of theperformance of the digital character by the human operator isfacilitated such that performance of the digital character isconcurrently driven by both the human operator and AI game-enginetechnology.

At block 608, according to a particular embodiment, a request isreceived from the customer. For example, with reference to FIG. 1, acustomer 110 may request that a box of cookies be provided.

At block 610, the request may be serviced by causing a change in thephysical environment in which the customer is located. By way ofexample, the request may be serviced by providing a physical object tobe delivered for retrieval by the customer. For example, with referenceto FIG. 5, the box 302 containing cookies is provided to be deliveredfor retrieval by the customer 110.

In selected embodiments, features and aspects described herein may beimplemented within a computing environment 700, as shown in FIG. 7,which may include one or more computer servers 701. The server 701 maybe operatively coupled to one or more data stores 702 (e.g., databases,indexes, files, or other data structures). The server 701 may connect toa data communication network 703 including a local area network (LAN), awide area network (WAN) (e.g., the Internet), a telephone network, asatellite or wireless communication network, or some combination ofthese or similar networks.

One or more client devices 704, 705, 706, 707, 708, 709, 710 may be incommunication with the server 701, and a corresponding data store 702via the data communication network 703. Such client devices 704, 705,706, 707, 708, 709, 710 may include, for example, one or more laptopcomputers 707, desktop computers 704, smartphones and mobile phones 705,tablet computers 706, televisions 708, motion capture sensor(s) 709,camera(s) 710, or combinations thereof. In operation, such clientdevices 704, 705, 706, 707, 708, 709, 710 may send and receive data orinstructions to or from the server 701 in response to user inputreceived from user input devices or other input. In response, the server701 may serve data from the data store 702, alter data within the datastore 702, add data to the data store 702, or the like, or combinationsthereof.

In selected embodiments, the server 701 may transmit one or more mediafiles including audio and/or video content, encoded data, generateddata, and/or metadata from the data store 702 to one or more of theclient devices 704, 705, 706, 707, 708, 709, 710 via the datacommunication network 703. The devices may output the audio and/or videocontent from the media file using a display screen, projector, or otherdisplay output device. In certain embodiments, the system 700 configuredin accordance with features and aspects described herein may beconfigured to operate within or support a cloud computing environment.For example, a portion of, or all of, the data store 702 and server 701may reside in a cloud server.

With reference to FIG. 8, an illustration of an example computer 800 isprovided. One or more of the devices 704, 705, 706, 707, 708 of thesystem 700 may be configured as or include such a computer 800.

In selected embodiments, the computer 800 may include a bus 803 (ormultiple buses) or other communication mechanism, a processor 801, mainmemory 804, read only memory (ROM) 805, one or more additional storagedevices 806, and/or a communication interface 802, or the like orsub-combinations thereof. Embodiments described herein may beimplemented within one or more application specific integrated circuits(ASICs), digital signal processors (DSPs), digital signal processingdevices (DSPDs), programmable logic devices (PLDs), field programmablegate arrays (FPGAs), processors, controllers, micro-controllers,microprocessors, other electronic units designed to perform thefunctions described herein, or a selective combination thereof. In allembodiments, the various components described herein may be implementedas a single component, or alternatively may be implemented in variousseparate components.

The bus 803 or other communication mechanism, including multiple suchbuses or mechanisms, may support communication of information within thecomputer 800. The processor 801 may be connected to the bus 803 andprocess information. In selected embodiments, the processor 801 may be aspecialized or dedicated microprocessor configured to perform particulartasks in accordance with the features and aspects described herein byexecuting machine-readable software code defining the particular tasks.Main memory 804 (e.g., random access memory—or RAM—or other dynamicstorage device) may be connected to the bus 803 and store informationand instructions to be executed by the processor 801. Main memory 804may also store temporary variables or other intermediate informationduring execution of such instructions.

ROM 805 or some other static storage device may be connected to a bus803 and store static information and instructions for the processor 801.The additional storage device 806 (e.g., a magnetic disk, optical disk,memory card, or the like) may be connected to the bus 803. The mainmemory 804, ROM 805, and the additional storage device 806 may include anon-transitory computer-readable medium holding information,instructions, or some combination thereof—for example, instructionsthat, when executed by the processor 801, cause the computer 800 toperform one or more operations of a method as described herein. Thecommunication interface 802 may also be connected to the bus 803. Acommunication interface 802 may provide or support two-way datacommunication between the computer 800 and one or more external devices(e.g., other devices contained within the computing environment).

In selected embodiments, the computer 800 may be connected (e.g., viathe bus 803) to a display 807. The display 807 may use any suitablemechanism to communicate information to a user of a computer 800. Forexample, the display 807 may include or utilize a liquid crystal display(LCD), light emitting diode (LED) display, projector, or other displaydevice to present information to a user of the computer 800 in a visualdisplay. One or more input devices 808 (e.g., an alphanumeric keyboard,mouse, microphone) may be connected to the bus 803 to communicateinformation and commands to the computer 800. In selected embodiments,one input device 808 may provide or support control over the positioningof a cursor to allow for selection and execution of various objects,files, programs, and the like provided by the computer 800 and displayedby the display 807.

The computer 800 may be used to transmit, receive, decode, display, etc.one or more video files. In selected embodiments, such transmitting,receiving, decoding, and displaying may be in response to the processor801 executing one or more sequences of one or more instructionscontained in main memory 804. Such instructions may be read into mainmemory 804 from another non-transitory computer-readable medium (e.g., astorage device).

Execution of sequences of instructions contained in main memory 804 maycause the processor 801 to perform one or more of the procedures orsteps described herein. In selected embodiments, one or more processorsin a multi-processing arrangement may also be employed to executesequences of instructions contained in main memory 804. Alternatively,or in addition thereto, firmware may be used in place of, or inconnection with, software instructions to implement procedures or stepsin accordance with the features and aspects described herein. Thus,embodiments in accordance with features and aspects described herein maynot be limited to any specific combination of hardware circuitry andsoftware.

Non-transitory computer readable medium may refer to any medium thatparticipates in holding instructions for execution by the processor 801,or that stores data for processing by a computer, and include allcomputer-readable media, with the sole exception being a transitory,propagating signal. Such a non-transitory computer readable medium mayinclude, but is not limited to, non-volatile media, volatile media, andtemporary storage media (e.g., cache memory). Non-volatile media mayinclude optical or magnetic disks, such as an additional storage device.Volatile media may include dynamic memory, such as main memory. Commonforms of non-transitory computer-readable media may include, forexample, a hard disk, a floppy disk, magnetic tape, or any othermagnetic medium, a CD-ROM, DVD, Blu-ray or other optical medium, RAM,PROM, EPROM, FLASH-EPROM, any other memory card, chip, or cartridge, orany other memory medium from which a computer can read.

In selected embodiments, the communication interface 802 may provide orsupport external, two-way data communication to or via a network link.For example, the communication interface 802 may be a wireless networkinterface controller or a cellular radio providing a data communicationnetwork connection. Alternatively, the communication interface 802 mayinclude a LAN card providing a data communication connection to acompatible LAN. In any such embodiment, the communication interface 802may send and receive electrical, electromagnetic, or optical signalsconveying information.

A network link may provide data communication through one or morenetworks to other data devices (e.g., client devices as shown in thecomputing environment 700). For example, a network link may provide aconnection through a local network of a host computer or to dataequipment operated by an Internet Service Provider (ISP). An ISP may, inturn, provide data communication services through the Internet.Accordingly, a computer 800 may send and receive commands, data, orcombinations thereof, including program code, through one or morenetworks, a network link, and communication interface 802. Thus, thecomputer 800 may interface or otherwise communicate with a remote server(e.g., server 701), or some combination thereof.

The various devices, modules, terminals, and the like described hereinmay be implemented on a computer by execution of software comprisingmachine instructions read from computer-readable medium, as discussedabove. In certain embodiments, several hardware aspects may beimplemented using a single computer; in other embodiments, multiplecomputers, input/output systems and hardware may be used to implementthe system.

For a software implementation, certain embodiments described herein maybe implemented with separate software modules, such as procedures andfunctions, each of which performs one or more of the functions andoperations described herein. The software codes can be implemented witha software application written in any suitable programming language andmay be stored in memory and executed by a controller or processor.

The foregoing described embodiments and features are merely exemplaryand are not to be construed as limiting the present invention. Thepresent teachings can be readily applied to other types of apparatusesand processes. The description of such embodiments is intended to beillustrative, and not to limit the scope of the claims. Manyalternatives, modifications, and variations will be apparent to thoseskilled in the art.

What is claimed is:
 1. A method for controlling performance of a digitalcharacter depicted at a display device located in a physicalenvironment, the method comprising: determining a presence of a firstperson located in the physical environment; in response to determiningthe presence of the first person, facilitating control of theperformance of the digital character depicted at the display device by asecond person and an artificial intelligence (AI) game-engine incombination, wherein determining the presence of the first personcomprises autonomously determining at least one of a characteristic ofan appearance of the first person, an emotional state of the firstperson, or an audio originating from the first person, and whereinfacilitating control of the performance of the digital character by thesecond person comprises providing at least one option selectable by thesecond person, for controlling the digital character to, in aninteraction with the first person, address the first person according tothe determined characteristic or the determined emotional state of thefirst person; and during the interaction with the first person, causinga change in the physical environment in which the first person islocated, in response to an action by the digital character depicted atthe display device, wherein the change in the physical environmentcorresponds to a physical object different from the display device. 2.The method of claim 1, wherein the determination of the presence of thefirst person is performed based on data received from at least a cameraor a microphone located in the physical environment.
 3. The method ofclaim 1, wherein facilitating control of the performance of the digitalcharacter by the second person comprises receiving motion capture datacorresponding to the second person.
 4. The method of claim 3, whereinreceiving the motion capture data comprises: receiving data from anoptical motion capture system; and receiving data from an inertialmotion capture system.
 5. The method of claim 4, wherein the datareceived from the optical motion capture system comprises datacorresponding to one or more optical markers located at a waist regionof the second person.
 6. The method of claim 4, wherein the datareceived from the optical motion capture system is for performing driftcorrection of a position of the second person that is determined basedon the data received from the inertial motion capture system.
 7. Themethod of claim 1, wherein the control of the performance of the digitalcharacter by the second person is facilitated such that performance ofthe digital character is concurrently driven by both the second personand the AI game-engine.
 8. The method of claim 1, further comprising:receiving a request from the first person; and servicing the request bycausing a change in the physical environment in which the first personis located, wherein servicing the request comprises providing a physicalobject to be delivered for retrieval by the first person.
 9. Anapparatus for controlling performance of a digital character depicted ata display device located in a physical environment, the apparatuscomprising: a network communication unit configured to transmit andreceive data; and one or more controllers configured to: determine apresence of a first person located in the physical environment; inresponse to determining the presence of the first person, facilitatecontrol of the performance of the digital character depicted at thedisplay device by a second person and an artificial intelligence (AI)game-engine in combination, wherein the one or more controllers arefurther configured to determine the presence of the first person byautonomously determining at least one of a characteristic of anappearance of the first person, an emotional state of the first person,or an audio originating from the first person, and wherein the one ormore controllers are further configured to facilitate control of theperformance of the digital character by the second person by providingat least one option selectable by the second person, for controlling thedigital character to, in an interaction with the first person, addressthe first person according to the determined characteristic or thedetermined emotional state of the first person; and during theinteraction with the first person, cause a change in the physicalenvironment in which the first person is located, in response to anaction by the digital character depicted at the display device, whereinthe change in the physical environment corresponds to a physical objectdifferent from the display device.
 10. The apparatus of claim 9, whereinthe determination of the presence of the first person is performed basedon data received from at least a camera or a microphone located in thephysical environment.
 11. The apparatus of claim 9, wherein the one ormore controllers are configured to facilitate control of the performanceof the digital character by the second person by receiving motioncapture data corresponding to the second person.
 12. The apparatus ofclaim 11, wherein receiving the motion capture data comprises: receivingdata from an optical motion capture system; and receiving data from aninertial motion capture system.
 13. The apparatus of claim 12, whereinthe data received from the optical motion capture system comprises datacorresponding to one or more optical markers located at a waist regionof the second person.
 14. The apparatus of claim 12, wherein the datareceived from the optical motion capture system is for performing driftcorrection of a position of the second person that is determined basedon the data received from the inertial motion capture system.
 15. Theapparatus of claim 9, wherein the control of the performance of thedigital character by the second person is facilitated such thatperformance of the digital character is concurrently driven by both thesecond person and the AI game-engine.
 16. A machine-readablenon-transitory medium having stored thereon machine-executableinstructions for controlling performance of a digital character depictedat a display device located in a physical environment, the instructionscomprising: determining a presence of a first person located in thephysical environment; in response to determining the presence of thefirst person, facilitating control of the performance of the digitalcharacter depicted at the display device by a second person and anartificial intelligence (AI) game-engine in combination, whereindetermining the presence of the first person comprises autonomouslydetermining at least one of a characteristic of an appearance of thefirst person, an emotional state of the first person, or an audiooriginating from the first person, and wherein facilitating control ofthe performance of the digital character by the second person comprisesproviding at least one option selectable by the second person, forcontrolling the digital character to, in an interaction with the firstperson, address the first person according to the determinedcharacteristic or the determined emotional state of the first person;and during the interaction with the first person, causing a change inthe physical environment in which the first person is located, inresponse to an action by the digital character depicted at the displaydevice, wherein the change in the physical environment corresponds to aphysical object different from the display device.